Skip to content

Nightly build is failing due version conflict between storage datalak… - #8635

Merged
praveenkuttappan merged 3 commits into
Azure:masterfrom
praveenkuttappan:globaltestissue
Nov 14, 2019
Merged

Nightly build is failing due version conflict between storage datalak…#8635
praveenkuttappan merged 3 commits into
Azure:masterfrom
praveenkuttappan:globaltestissue

Conversation

@praveenkuttappan

Copy link
Copy Markdown
Member

Azure-storage-file-datalake has dependency to 12.0.0 as per setup.py and causing to install 12.0.0 version of azure-storage-blob during nightly build global test instead of latest 12.1.0b1 and causing test failures for new test cases added for azure-storage-blob.

Fix is to update azure-storage-blob dependency to >=12.0.0 in azure-storage-file-datalake

@adxsdk6

adxsdk6 commented Nov 14, 2019

Copy link
Copy Markdown

Can one of the admins verify this patch?

@praveenkuttappan

Copy link
Copy Markdown
Member Author

@xiafu-msft , @rakshith91 , @lmazuel : Can you please take a look and let me know if this change in setup.py to update the dependency is fine. Currently nightly build is failing due to this dependency against azure-storage-blob 12.0.0 version and current dev version is 12.1.0b1

@xiafu-msft

xiafu-msft commented Nov 14, 2019

Copy link
Copy Markdown
Contributor

I think we don't want to change the version to "azure-storage-blob>=12.0.0". That is super dangerous when there is breaking changes in azure-storage-blob>=13.0.0
lets add an upper bound to make sure the major version is less than 13

Comment thread sdk/storage/azure-storage-file-datalake/setup.py Outdated
Comment thread sdk/storage/azure-storage-file-datalake/setup.py
@praveenkuttappan
praveenkuttappan merged commit 5761ad4 into Azure:master Nov 14, 2019
@praveenkuttappan
praveenkuttappan deleted the globaltestissue branch November 14, 2019 23:54
"azure-core<2.0.0,>=1.0.0",
"msrest>=0.6.10",
"azure-storage-blob~=12.0.0"
"azure-storage-blob<13.0.0,>=12.0.0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the upper bound!😊
While I still have some concern:
1)is 12.0.0b1 > 12.0.0? I’m not very sure(I’m worried this commit is not fixing the problem)
2) What’s the purpose of nightly build? Is it to run test for all packages in live? Currently azure-storage-file-datalake depends on exactly bob 12.0.0, so 12.0.0b1 will break the tests for datalake package. Because there are a lot of breaking changes between 12.0.0 and 12.0.0b1
3) azure-storage-blob is GA now, why are we still using 12.0.0b1?

Comment thread shared_requirements.txt
#override azure-storage-file-share msrest>=0.6.10
#override azure-storage-file-datalake msrest>=0.6.10
#override azure-storage-file-datalake azure-storage-blob~=12.0.0
#override azure-storage-file-datalake azure-storage-blob~=12.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it necessary to touch the version here?
Could we upgrade the version of nightly build to 12.0.0 instead of using the preview version?

@xiafu-msft

Copy link
Copy Markdown
Contributor

Sorry that the comment wasn’t sent out successfully.. I didn’t notice that before you merge 😂

@xiafu-msft

Copy link
Copy Markdown
Contributor

I think the commit probably will not fix the nightly build?
azure-storage-file-datalake is only compatible with azure-storage-blob version 12.0.0. azure-storage-file-datalake will be broken if you force it to depend on 12.0.0b1

@scbedd

scbedd commented Nov 15, 2019

Copy link
Copy Markdown
Member

Given that:

  • The latest azure-blob will be installed BEFORE we install your package,
  • The version specifier has been relaxed to allow floating minor.

Here's the relevant section from the packaging docs:

Adopting this approach as a project author allows users to make use of “compatible release” specifiers, where name ~= X.Y requires at least release X.Y, but also allows any later release with a matching MAJOR version.

So your package shouldn't eliminate the dev version of blob that was installed prior. Does this make sense?

Praveen has also run a local test run to confirm Laurent's suggestion :)

If it doesn't work, we will definitely adjust and try again!

xiangyan99 added a commit that referenced this pull request Nov 15, 2019
* InteractiveBrowserCredential prompts for account selection (#8470)

* remove *.yaml under tests folder from manifest.in (#8625)

* [AutoPR] cosmos-db/resource-manager (#8560)

* regenerated

* fix script

* another place

* fix

* fix

* one more change

* one more

* undo multiapi script

* regeneated

* undo multiapi script

* history, version + additional fix

* regenerated

* fixed test

* fixed generation mistake

* added more breaking changes to history

* regenerated again

* additional fixes

* udpated test names

* test recordings

* Update README.md (#8628)

Add standard section on security issues below the "file an issue via GitHub Issues' bullet.

* Handle exceptions when constructing DefaultAzureCredential (#8294)

* Python 3.8 test pipeline is not cancelled when build is cancelled (#8544)

* Skip python 3.8 testing when CI is cancelled

* Cosmos docstring review (#8607)

* cosmos docstring edits

* user-defined

* trailing whitespace

* xfail flaky emulator tests

* review feedback

* Default credentials are configurable by kwargs (#8514)

* Nightly build is failing due version conflict between storage datalak… (#8635)

* Nightly build is failing due version conflict between storage datalake and storage blob

* InkRecognizer to use GA azure-core (#8341)

* InkRecognizer to use Ga azure-core

* Disable auto-pr update

* Azconfig remove tests from whl (#8663)

* update tests

* ignoring the azure folder directly (#8673)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants